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ABSTRACT 

This paper discusses the value of systems studies that 
provide unbiased “honest broker” assessments of the 
quantified benefits afforded by advanced technologies for 
specific missions. The organization, format and approach 
used by the NASA Glenn Research Center (GRC) 
Systems Assessment Team (SAT) to perform system 
studies for the GRC advanced power and on-board 
propulsion technology development program is 
described. Three levels of assessments and a sensitivity 
analysis are explained and example results are 
presented. The impact of system studies results and 
some of the main challenges associated with systems 
studies are identified. A call for collaboration is made 
where system studies of all types from all organizations 
can be reviewed, providing a forum for the widest peer 
review to ensure accurate and unbiased technical 
content, and to avoid needless duplication. 

INTRODUCTION 

System studies provide the basis for sound technology 
program planning. The results of the tradeoff studies and 
technical assessments performed in system studies help 
provide the rationale to advocate and defend technology 
development. In its broadest application, system studies 
indicate the range of applicability of various technologies. 
More specifically, a system study can indicate the most 
beneficial technology for a given mission or application. 

In the past, a successful case for a competing 
technology could be made by simply stating that it would 
cost less, weigh less or be more reliable than the 
baseline (or faster, better, cheaper, if you will). Now, in 
order to better justify the expense, technology program 
planners and financiers demand to know the magnitude 
of the technological benefit for specific missions or 
applications. 

In many cases, the results of a system study do not 
indicate a clear choice of technology for a mission or 
application. One technology may result in the lowest 
mass yet have the highest development or non-recurring 


cost. Also, systems studies performed by different 
groups can lead to apparently contradictory results 
because the groups make slightly different assumptions 
for a number of parameters for different reasons, or use 
different criteria to assess the results. Assessment 
results can be further skewed if they are based on over 
optimistic predictions of the performance of newer, less 
mature technologies. Frequently, and for a variety of 
reasons, the actual performance of a newer technology 
in a specific application is much more conservative than 
initial optimistic expectations. 

In short, a well-planned and defendable technology 
development program needs accurate, consistent, 
unbiased “honest broker” technology assessments that 
temper the enthusiasm of advocates and fairly quantifies 
the technology benefits for specific missions and 
applications. It is also desirable to eliminate duplication 
and repetition in performing the systems studies in order 
make the best use of human resources. 

GRC SYSTEM ASSESSMENT TEAM 

At the NASA Glenn Research Center, the System 
Assessment Team (SAT) has been formed to coordinate 
the performance of systems studies for the space power 
and on-board propulsion technology development 
programs. The SAT is intended to be a forum to 
coordinate internal peer review of technology trade 
studies, system assessments, and preliminary mission 
conceptual design and system sizing. 

ORGANIZATION 

Systems analysts from project, technology and cost 
organizations at GRC comprise the SAT. Team 
membership is open, with participants joining the group 
as necessary - i.e. when their work involves some sort of 
technology assessment or trade study that warrants 
wider coordination and review. 

The core members of the SAT are matched up with task 
managers from the current GRC power and on-board 
propulsion technology development program. While the 
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technology task managers are experts in their specific 
technology, they may not always be aware of the entire 
mission set or range of applications that their technology 
can benefit. Also, they may not have the analytical tools 
to quantify the mission or application benefit. The SAT 
systems analysts work closely with the technology task 
managers to ensure that the entire range of applicability 
of a given technology is identified, the features and 
benefits of the technology are documented, and the 
quantification of the technology benefits for specific 
missions and applications is performed. Currently, the 
SAT has analysts for each of the following specific 
technology areas: 

• Photovoltaics - high efficiency solar cells, thin-film 
cells and advanced photovoltaic concepts such as 
quantum dot cells. 

• Fuel Cells - active and passive regenerative fuel 
cells; “unitized” stacks. 

• Chemical Batteries - Nickel-Hydrogen, Nickel Metal 
Hydride, Lithium Ion. 

• Aerospace Flywheels - advanced concepts for 
energy storage and combined energy storage and 
spacecraft attitude control. 

• Stirling Radioisotope Power System - free-piston 
Stirling thermal-to-electric energy conversion. 

• A dvanced Power Conditioning, Management & 
Control Components - high efficiency converters 
with digital control; low and high temperature 
operation; power electronic building blocks. 

• Low Temperature Electronics 

• Advanced Electrochemical Capacitors - high specific 
power for peak power applications. 

• Power System Surface Coatings & Materials - 
atomic oxygen resistant solar array coatings; 
intercalated graphite EMI shielding; thermal control 
surfaces and coatings. 

• Power System Environmental Design Codes - 
models for spacecraft charging, grounding and solar 
array arc mitigation. 

Although the benefit of a specific power subsystem 
technology - power generation, energy storage and 
power management and distribution, or PMAD - can be 
quantified on its own, a more complete systems level 
assessment will indicate the full magnitude of the 
technology benefit for a given mission. Therefore, 
another SAT analyst assesses the mission benefit of the 
combination of advanced power generation, energy 
storage, and power management and distribution 
technologies. Even more, the synergistic benefits of 
advanced power and on-board electrical propulsion 
technologies warrant specific consideration. This 
analyst’s approach also includes sensitivity studies to 
determine which parameters have the largest impact on 
the full system. 


FORMAT 

The SAT meets weekly or bi-weekly to discuss normal 
business - discuss current events, monitor team 
member’s progress on various studies, and plan future 
activities. Occasionally, there will be a focused peer 
review of an analysts’ system study results, usually prior 
to an external briefing. Periodically, special seminars are 
held with technologists in order to review the latest 
technology developments and obtain updated values for 
some of the key technology performance parameters 
that drive the quantitative analysis. Quarterly briefings 
are held with the GRC power and on-board propulsion 
program managers to provide them the systems studies 
results they need to guide their program planning. The 
program managers also provide feedback on the types of 
analyses they would like to see performed. 

APPROACH 

An assessment performed in a system study can take a 
variety of forms and be performed at a number of levels. 
Most assessments take the form of a trade study where 
some aspect, typically mass, of alternative subsystem 
technologies are calculated and compared for a specific 
mission or application. The mission requirements must 
first be known or assumed so that the subsystems can 
be sized. Depending on the need for more or less detail 
and the time spent on the study, system sizing or 
performance models of varying levels of complexity are 
used. The following subsections discuss and illustrate 
some of the more common types of assessments. 

Top-Level Assessments 

Top-level assessments are used for rapid turnaround of 
very preliminary studies where only general trends are 
important. They are also used for “what if’ assessments 
where a given level of performance is simply assumed 
so that the corresponding benefits can be quickly 
quantified. It then remains for the analyst or program 
manager to explain how this level of performance will be 
obtained. 

An example of a top-level preliminary assessment 
showing the potential mission benefits of advanced 
power and on-board electrical propulsion can be found in 
[1], This assessment is top-level in that it uses a model 
that operates with assumptions for the top-level 
performance parameters for each power subsystem 
(specific power, W/kg for solar arrays and specific 
energy, WH/kg for energy storage, etc.). It also does not 
determine whether or not the advanced power and 
propulsion systems could have been actually 
implemented on the spacecraft studied, or that the 
benefits would have been actually realized (i.e. would the 
spacecraft actually flown more payload instruments?). 
However, the model does account for all major elements 
of the power and electric propulsion subsystems. 
Further, it was sophisticated enough to account for the 
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fact that if more mass was allocated to the payload, the 
increased payload would require more power. The 
model used a feedback loop to iterate on the final 
spacecraft mass breakdown. 

A unique feature of this analysis is that, for a variety of 
NASA missions, it demonstrates a truly synergistic effect 
- the spacecraft mass benefit of using electric propulsion 
together with advanced power is much greater than the 
mass benefit using advanced technology in any portion 
of the power and propulsion subsystems alone. 

Figure 1 illustrates such results for the low Earth orbit 
Tropical Rainfall Measurement Mission (TRMM) 
spacecraft. The chart shows a 31% increase in the mass 
of the “balance-of-spacecraft” is possible if high- 
efficiency solar arrays and flywheel energy storage is 
used in conjunction with low power Hall effect electric 
propulsion. The benefit of using these advanced power 
or propulsion technologies alone is a 20% increase in the 
balance-of-spacecraft mass. The balance-of-spacecraft 
includes the payload and subsystems other than power 
and propulsion. 
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Figure 1 - Top-level model results showing the 
synergistic benefit of advanced power and electric 
propulsion for the TRMM spacecraft. 

Detailed Preliminary Assessments 

Given more time and the need for a more thorough 
study, detailed system assessments can be performed. 
In these cases, models using lower level parameters and 
algorithms that account for more detailed operation of 
power and propulsion systems must be developed or 
adapted and applied to specific missions. This takes time 
and requires knowledge or assumptions of more of the 
requirements regarding the mission, spacecraft and 
environment as well as design features of the power and 
propulsion subsystems in order to feed the more detailed 
models. If the time is available to perform such a study, 
and the model adequately benchmarked, there should be 
a higher level of confidence that the quantified results 
and trends are meaningful and realizable. However, the 


assessment should still be considered preliminary in that 
only a full-blown detailed design study will determine the 
proper characteristics of an actual system. 

An example of a preliminary, yet detailed assessment is 
given in [2], In this study, detailed power and solar 
electric propulsion computer models were used to size 
power systems for a 1 megawatt class Human Mars 
Mission Solar Electric Propulsion (SEP) stage and a 
smaller 15 kW Europa Mapper spacecraft. For both 
missions, the size (deployed area and mass) of a 
number of photovoltaic solar array power generation 
concepts was calculated using a time-dependent power 
system performance model iteratively coupled with the 
time-dependent trajectory model. Detailed solar array 
performance was modeled throughout each mission. The 
results of the study were used to recommend baseline 
power system options for these future missions. 

For the Mars mission analyzed in this study, figure 2 
shows the power required (dashed line) by the electric 
propulsion power processing units (PPUs) and the 
performance of a solar array using thin-film amorphous 
silicon/germanium solar cells as a function of mission 
time. The detailed electrical power system performance 
model used in this study accounts for all significant 
orbital environment effects, including thermal, particulate 
radiation, meteoroid/orbital debris and plasma. Note the 
rapid degradation in solar array power as the SEP stage 
spirals through the proton belts during the first 200 days 
of the mission. 


Total Power to PPUs at 500 V 



Figure 2 - Detailed power system performance 
modeling showing array performance over time. 


Intermediate-Level Assessments 

One step beyond top-level assessments, yet not as 
sophisticated as detailed assessments, are intermediate 
level, or first order assessments. These assessments 
use models that account for all significant phenomena 
and parameters that affect power and propulsion system 
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performance, at least to a first order. They are similar to 
detailed assessments in that overall power system 
performance, say in terms of specific power (w/kg), is a 
calculated value based on more detailed component 
models. They are similar to top-level assessments in that 
some parameters that effect power system sizing are 
best estimates for the mission under study, or linearly 
scaled from baseline system data. Intermediate-level 
assessments can be performed relatively quickly yet 
contain enough detail to better distinguish between 
closely competing technology options. They also can be 
used to perform rapid parametric analysis to discern 
trends among competing technology options. 

An example of a first-order model and intermediate-level 
assessment is found in [3]. This reference describes a 
model developed to calculate size and estimate the 
relative cost of photovoltaic solar arrays for a variety of 
NASA missions. The model was applied in a parametric 
fashion to quantitatively determine the mass and cost 
trends among arrays using higher efficiency, yet heavier 
crystalline solar cell technology, versus arrays using less 
efficient, but much lighter thin-film solar cell technology 
deposited on lightweight flexible polyimide substrates. 


(Mn Belt AsteroWToir Solar Any (9 kW 9 1 AU 2TQ with a Hejdble Si*»»rae 
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Figure 3 - Intermediate-level parametric analysis 
array mass predictions using a first-order array 
assessment model. 

Figure 3 shows that for the given mission requirements, 
a solar array with 12% efficient lightweight thin-film solar 
cells (copper-indium-disulfide or CIS2) on a polyimide 
substrate will have the same mass as an array with 
either 20% efficient thin silicon crystalline cells (Thin Si) 
or 35% efficient four-junction gallium arsenide (GaAs 4-j) 
based crystalline cells. While the goal of the thin-film cell 
technology developers is greater than 20% efficiency in 
order to mitigate solar array deployed area impacts, this 
system study indicated that thin-film cells with 


intermediate levels of efficiency (-12%) will be 
competitive from a mass standpoint. 

Sensitivity Analysis 


Although it could be considered a top-level or first-order 
intermediate assessment, a sensitivity analysis can be 
used to determine the relative importance of various 
component or subsystem parameters with respect to a 
key system or spacecraft-level parameter. In this case, 
the input parameters of a top-level or first order power or 
propulsion system model are varied, one at a time, while 
every other input parameter is held constant. The impact 
on system mass for each parameter variation is 
calculated. The most sensitive input parameter is the one 
that results in the largest percent change in the system 
mass per percent change in input parameter value. 


For example, in a recent sensitivity analysis, the solar 
array specific power (W/kg), the energy storage specific 
energy (WH/kg) and the power management and 
distribution (PMAD) specific power (W/kg) input 
parameters in a top-level power system sizing model 
were varied from their nominal baseline values for a Low 
Earth Orbit (LEO) science mission. 


Figure 4 shows the normalized results. For this LEO 
mission, the largest decrease in total power system 
mass occurs when the specific energy of the baseline 
chemical battery energy storage system improves. The 
model indicates that a factor of two improvement in 
specific energy will decrease the total mass of the power 
system by 1 8%, everything else held constant. 



Figure 4 - The use of sensitivity analysis to 
determine the system-level impact of subsystem 
variations. 
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There are a number of issues associated with this type of 
analysis, which are in part dependent on the level of 
sophistication of the underlying model. First of all, while a 
decent model will account for the interdependence of key 
parameters, some input parameters should not be varied 
independently. For example, significantly changing the 
value of specific energy for a chemical battery energy 
storage system implies switching to an alternative battery 
technology, which would in turn imply a different watt- 
hour (“roundtrip”) efficiency. Simply varying the battery 
specific energy without changing the watt-hour efficiency 
may not be appropriate. To properly account for this, 
both parameters should be varied together. 

Second, the results of a particular sensitivity analysis will 
only be applicable within a specific range of the nominal 
baseline mission values. Also, for power systems, the 
most sensitive parameter for a mission requiring 
significant amounts of energy storage may be completely 
different than the parameter for missions requiring no 
energy storage. 

However, used carefully, the results from a thorough 
sensitivity analysis can be used to discern meaningful 
trends and set technology development priorities with 
respect to a given criterion (e.g. power system mass). 

IMPACT 

The results of systems studies from all levels of 
assessment can serve to identify the need for new 
technology development, identify a new application of an 
existing technology, further strengthen the case for an 
existing technology and/or application, indicate where 
additional systems studies should be performed, or flag 
where program re-planning should occur. The quantified 
benefits for specific missions provide defensible rationale 
for program advocacy and defense and can be included 
in proposals for new development. 

CHALLENGES 

Systems analysts strive to develop accurate models and 
perform quantitative assessments from which they can 
provide definitive recommendations to program and 
mission planners. However, if the analyst took the effort 
and care to estimate (quantitatively or qualitatively) and 
disclose the uncertainty associated with most preliminary 
assessments, the less definitive the trade study results 
and recommendations would become. In fact, trade 
studies performed by independent analysts from different 
organizations will often contain different results due to an 
accumulation in assumption differences, slightly different 
modeling techniques, and different weighting of the 
decision criteria. This does not necessarily mean that a 
given analyst is biased or that preliminary system study 
results are meaningless. It simply highlights the point that 
for preliminary top-level assessments, mission 
requirements and system performance are not well 
defined and there are a number of different ways to look 
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at and evaluate a technology. Care must be taken to not 
decide “winning” and “losing” technologies on the basis 
of a preliminary study that looked at mass or cost alone. 
A more thorough, detailed assessment could determine 
whether or not the potential benefits are indeed 
realizable and help quantify other factors, such as 
system reliability or system integration and operation. 
The detailed study results could lead to much different 
conclusions than those derived from a preliminary 
assessment. Some other specific challenges are 
discussed below. 

As just mentioned, more detailed mission and baseline 
system data is needed in order to better determine 
whether or not the preliminary benefits are actually 
realizable or to size an alternative system base on 
advanced technology. However, this detailed mission 
and system data is not always available or is not easily 
obtained by technology development organizations. 

Also, a fair comparison of the performance of existing, 
mature technology with the projected performance of 
newer technologies is a significant challenge. Regarding 
the new technologies, there is not yet a database of 
actual performance from which to make projections. In 
these cases, the newer technologies almost always look 
favorable as compared with the mature technologies. 
One way to more fully characterize the potential 
performance of the newer technologies is through 
parametric analysis. In this approach, the performance of 
the newer technology is assumed through a range of 
more conservative (i.e. less optimistic) values than that 
that espoused by the advocates. In this way, the 
advantage of the newer technology can be assessed 
across a range of possibilities, including those that fall 
short of initial expectations. 

Technology program planners sometimes seem to prefer 
quantified technology benefits for specific missions as 
opposed to benefits for representative missions or 
mission classes. The challenge then to an analyst 
evaluating a given technology is figure out how to 
quantify what benefit for how many missions of what type 
in order to provide sufficient rationale to justify 
technology development. The quantified mission benefits 
may be very meaningful to one mission planner, yet 
irrelevant to another. This requires the system analyst 
perform a large number of quantified mission benefit 
studies of different mission types to ensure relevancy to 
the largest number of mission planners. 

Finally, most decisions boil down to one of cost. 
However, advanced development and non-recurring 
costs, let alone flight hardware recurring costs are very 
hard to estimate for immature technologies. One way to 
mitigate the uncertainty associated with the absolute cost 
of a technology is to perform relative costs estimates. In 
this case, if the absolute cost relationships are 
inaccurate, at least all options are costed using the same 
methods so that the relative costs may be reasonable. 
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COLLABORATION 

Peer-reviewed, unbiased assessments that fairly quantify 
the benefits of technologies for specific missions should 
be of interest to the technical community at large, as well 
as technology program and mission planners. Therefore, 
it would be beneficial to present system studies results 
from many different organizations. A special conference 
or other meeting would provide a useful venue for 
systems analysts to present to each other and to 
technology program planners and mission planners. 

The peer review in this forum would expose parochialism 
and overt salesmanship, ensuring the most technically 
accurate and fair assessments get the greatest 
recognition. Widespread dissemination of results would 
also help avoid the needless duplication of studies. 

SUMMARY 

This paper discussed the value of systems studies that 
provide unbiased “honest broker" assessment of the 
quantified benefits afforded by advanced technologies. 
The organization, format and approach used by the 
NASA GRC Systems Assessment Team to perform 
system studies for the GRC advanced power and on- 
board propulsion technology development program were 
outlined. Three levels of assessments were explained 
(Top-Level, Intermediate, and Detailed) and example 
results were presented. Sensitivity analysis used to 
analytically determine the most sensitive subsystem 
parameter was described. 

It was then stated that system study results can be used 
to identify the need for new technology development, 
identify a new application of an existing technology, 
further strengthen the case for an existing technology 
and/or application, indicate where additional systems 
studies should be performed, or flag where program re- 
planning should occur. 

Some of the main challenges associated with systems 
studies were identified and discussed, namely the 
uncertainty associated with the preliminary nature of the 
study, the lack of detailed mission and system 


information, the difficulty assessing the performance of 
mature versus new technology, the need to perform a 
large number of quantified mission benefit studies to 
ensure the widest relevancy, and the perennial problems 
associated with cost estimating. 

Finally a call for collaboration was made where system 
studies of all types from all organizations are reviewed. 
Here, the goal is to provide a venue for widespread 
communication and dissemination of results, a forum for 
the widest technical community peer review to ensure 
accurate and unbiased technical content, and to avoid 
needless duplication. 
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